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ABSTRACT: Recently, the popularity of 
timed-essay exams has increased, becoming part 
of the Graduate Management Admissions Test 
(GMAT) in the late 1990s and now being incorpo- 
rated into The College Board Scholastic Aptitude 
Test (SAT) in Spring of 2005 and ACT (American 
College Testing) test in Fall of 2004. This research 
evaluates the “value added” of an essay compo- 
nent, contrasting placement using ACT’S multiple 
choice COMPASS (Computerized Placement As- 
sessment Support System) writing test versus es- 
says holistically scored by English faculty. Evi- 
dence suggests (a) that combining the timed-es- 
say exam score with another score may improve 
accurate placement; (b) that the timed-essay ex- 
ams, not multiple-choice tests, may be fairer for 
minority students; (c) and that a questionnaire 
creates an invaluable context when relating scores 
on placement tests to final grades in courses. 

Since composition began as a field in the 
1960s, academics who specialize in composition 
and literacy have described why composition in- 
struction is inconsistent across the country in 
grades K through 12 and in higher education (Ber- 
lin, 1987; Bloom, Daiker, & White, 1996; Myers, 
1996; Strickland & Strickland, 1993). This “writ- 
ing on the wall" is larger now because of these 
headlines: “Writing in Schools Is Found Both Dis- 
mal and Neglected" in the April 26, 2003 New York 
Times (Lewin, 2003a); “Educators Demand Up- 
grade in Writing” in the May 13, 2003 Washington 
Post (Strauss); “2Rs Left in High School, Out of 
Choice or Fatigue, Many Teachers Have Aban- 
doned the Term Paper, Leaving a Hole in Col- 
lege-Bound Students' Education” in the May 19, 
2003 LA Times (Hayasaki). 

Although the April, 2003 report from The 
National Commission on Writing in America’s 
Schools and Colleges caused these headlines, per- 
haps composition professionals should have an- 
ticipated them because of other reports. Consider 
that, in 2000, the Education Commission of the 
States “released a report on improving teacher 
quality” and advocated for high schools and higher 
education to coordinate standards in subjects such 
as writing (Basinger, 2000). Consider too that, in 
2001, the National Commission on the High- 
School Senior Year issued this finding: Only 44% 
of freshmen have completed college preparatory 
curriculum in high school that positions them to 
succeed in college (Kellogg, 2001). 


Then, as 2003 approached, a 6-year national 
study directed by Stanford University was conclud- 
ing which claimed: Many states have a “disconnect 
between the requirements for graduation from 
public high schools and the requirements for ac- 
ceptance at state universities" (Rooney, 2003, p. 
1). In the words of one participating educator, high 
school graduates “come and take our placement 
test and they’re still in pre-college reading, writ- 
ing, and math” (Rooney, p. 2). 

Also in 2003, an 18-member commission, 
assembled by Pew Charitable Trusts and the Asso- 
ciation of American Universities, sent every high 
school in the country a booklet of education stan- 
dards— that includes standards for student writing — 
so that a “long overdue" link between “high-school 
and university standards" may be established (Pot- 
ter, 2003, p. 1). According to an English teacher 
participating in the development of the standards, 
presently the “standardized tests. ..don’t do an ef- 
fective job of measuring whether a student is mas- 
tering the process of writing" (Hebei, 2001, p. 2). 
All these reports imply that, as a nation, instruc- 
tors teach writing inconsistently if not poorly, but 
the reports also highlight the need and importance 
of accurate placement in college developmental 
English courses. 

Recently, placement tests for basic writing or 
composition courses have typically been either of 
these multiple-choice exams: the SAT or ACT 
(Abraham, 1986; Bernstein, 2001; Carter, 1991; 
Wiener, 1989). Beginning in 1901, the SAT was a 
written exam(s), but, as the influence of psycho- 
metricians grew in 1926, the SAT became a mul- 
tiple-choice test (Young, 2003, pp. 21-22). Between 
1977 and 1996, moreover, the College Board of- 
fered a SAT II test with a timed-essay section be- 
sides offering the SAT I test as a multiple-choice 
test (Young, 2003, p. 22). The current popularity 
of timed-essay exams is demonstrated by several 
events. In 1994, the exam became a mandatory 
part of the Graduate Management Admissions Test 
(GMAT) (Bridgeman & McHale, 1998). The timed- 
essay exam will be a mandatory part of SAT I in 
2005 (Hoover, 2002) and an optional part of the 
ACT test in 2004 (Young, 2002). These changes by 
the College Board and ACT, Inc. may be a response 
to pressure from Richard C. Atkinson, president 
of the University of California system, because that 
system has been the largest single consumer of SAT 
tests (Cavanagh, 2002; Hoover, 2002; Lewin, 
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2002b). It also may be related to proponents of 
direct writing assessment (Cooper, 1977; 
Greenberg, Wiener, & Donovan, 1986; Wolcott & 
Legg, 1998) who “have argued that multiple choice 
tests of writing ability are more reliable, but less 
valid, when compared to direct assessment proce- 
dures like holistic scoring [of timed essays]” 
(Williamson, 1994, p. 162). 

In general, typical means for placement into 
composition courses may range from scoring 
multiple-choice tests, to scoring timed-essay exams, 
to scoring portfolios. This study is primarily con- 
cerned with multiple-choice tests and timed-essay 
exams. However, this focus should not be inter- 
preted as advocating for either the timed-essay 
exam or multiple-choice test as the best way to 
place freshmen into writing courses. 

The main subject of this writing is to exam- 
ine the usefulness and accuracy of multiple-choice 
tests and timed-essay exams for student placement. 
In general, empirical research, English professors, 
and national organizations suggest that holistically 
scored, timed essays are a practical and ethical way 
to place freshmen into composition courses as 
compared to placement by an editing (or gram- 
mar) test and/or a reading test (i.e., “old" ACT 
and SAT tests). 

Placement Test Options 

In the field of composition, a timed-essay 
exam has been defined as preferable if only one 
measure for placement into composition courses 
will be used, and if the only alternative is a mul- 
tiple-choice test (Garrow, 1989; Hilgers, 1995; 
Jaskulek, 1986; Ravitch, 2002; White, 1995; Wolcott 
& Legg, 1998; Zinn 1988). Shedding light on this 
conclusion seems important because multiple- 
choice tests are still commonly used for placement 
into composition courses (Abraham, 1986; Carter, 
1991; Lalicker, 2001; Wiener, 1989). As a group, 
research reports suggest three categories of in- 
quiry: multiple-choice tests and predicting success 
in composition courses, implications of timed-es- 
say exams for program development, and multiple 
indicators as preferential for placement. 

Multiple-Choice Tests and Predicting 
Success in Composition Courses 

Researchers have asked this question: Are 
these multiple-choice tests— ACT composite, ACT 
English and/or reading sections and ACT'S AS- 
SET (Assessment of Skills for Successful Entry and 
Transfer) tests and SAT composite, SAT verbal 
and/or reading sections, and SAT APS (Assess- 
ment Program and Services) and MAPS (Multiple 
Assessment and Placement Services) tests— effec- 
tive placement tests for composition courses? Re- 
sponses became tied to whether these tests may 
predict final course grades. Some studies suggest 
that a predictive relationship does not exist 
(Behrman, 2000; College of the Canyons Office 
of Institutional Development, 1993; Gabe, 1989; 


Holderer, 1992; Isbell, 1988; Mathay, 1992; Meeker, 
1990; Whitcomb, 2003; Wonnacott, 1989). Other 
studies present mixed findings or support the 
notion that a predictive relationship does exist 
(Armstrong, 1994; College of the Canyons Office 
of Institutional Development, 1994; Escott, 1980; 
Hodges, 1990). 

Timed-Essay Exam Implications for 
Program Development 

Timed essays carry their own set of issues, 
such as the reliability of essay scores, biases of es- 
say prompts, and prescribed five-paragraph essay 
responses (Baron, 2003; Christenbury, 2002; 
Haviland & Clark, 1992; Hoover, 2002; Jaskulek, 
1986; Miraglia, 1995; Taylor, 1990; White, 1994; 
Winters, 1979; Wolcott & Legg, 1998). Test anxi- 
ety and time limitations may also negatively affect 
the reliability of timed-essay exams (Zinn, 1988). 
Nevertheless, when English teachers score (or rate) 
timed essays, scores have been found to have a 
predictive relationship with final grades and to 
be more indicative of students’ writing abilities 
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compared to multiple-choice test scores from the 
same students (Cunningham, 1983; Garrow, 1989; 
Holderer, 1992; Mathay, 1992). 

Using timed essays in a basic writing program, 
moreover, precipitates other benefits besides im- 
proved placement. Rating timed essays help En- 
glish teachers to “internalize the established crite- 
ria for the scoring program" and explore “how 
the criteria can be applied to students’ most re- 
cent writing efforts” (Wolcott & Legg, 1998, pp. 
60-61). Timed essays may also be used as pre- and 
posttests to examine whether student writers’ skills 
have improved (Cunningham, 1983; Drakulich, 
1979; Kiefer, 1983; Mills, 2000; Resch, 1972; 
Schaumburg, 1975; Strugala, 1983; Sweigart, 1996; 
Wolcott, 1996). When assessing a writing program, 
timed-essay scores may be considered along with 
program questionnaires (Strugala, 1983) and/or 
students’ portfolios (Wolcott, 1996). 

Multiple Indicators as Preferential for 
Placement 

Even though a timed-essay score may be a 
single or primary criterion for placement, multiple 
criteria may improve not only placement but also 
assessing writing courses or programs. For in- 
stance, some placement tests can be more predic- 
tive of final grades in writing courses when com- 


bined with the students’ high school GPAs 
(Armstrong, 1994; Cunningham, 1983; Whitcomb, 
2003). Also, timed-essay scores may be more pre- 
dictive of course grades or more reliable as an 
accurate placement instrument when combined 
with some multiple-choice test scores (Cummings, 
1991; Cunningham, 1983; Galbraith, 1986; 
Garrow, 1989; Isonio, 1994; Wolcott, 1996; Wolcott 
& Legg, 1998). Using more than one criterion for 
placement, moreover, may improve placing at-risk, 
first-generation, or minority students. Some stud- 
ies have found that minority students have been 
disproportionately affected by placement done 
with multiple-choice tests (Belcher, 1993; College 
of the Canyons Office of Institutional Develop- 
ment, 1996; Garrow, 1989; Jones & Jackson, 1991; 
White, 1990). Also, accurate placement— grouping 
basic or developmental writers and teaching them 
academic writing skills— may be key for success: 
Studies show that these students graduate in 
greater numbers than students who are not re- 
quired to be in basic writing programs or choose 
to ignore a recommendation to participate in such 
programs (Baker & Jolly, 1999; Boylan, 1999; 
Cunningham, 1983; Gleason, 2000; McGregor & 
Attinasi, 1996; White, 1994). 

Method 

Demographics 

In the Spring of 2003, the Office of Institu- 
tional Research, and English faculty began an 
evaluation of the writing placement process at an 
open-admission, 4-year, urban, public college in 
the western United States that has an average en- 
rollment of 24,000 students each year. An average 
of 710 students each fall semester and 565 each 
spring semester (1,275 total) enroll in basic writ- 
ing courses as a result of the placement system 
based on multiple-choice tests. Approximately half 
the college-wide student body requires one or more 
developmental courses. Also, college wide, stu- 
dents are about 92% European American, 51% 
part-time, and 43% female. 

A total of 431 students wrote a timed essay, 
of whom 358 (83%) remained in ENGH 0890 
(lower-level basic writing) or 0990 (upper-level basic 
writing) and received a final grade. Of these 358 
students, 289 (81%) completed the survey and 258 
(72%) were enrolled as full-time students (12+ 
credit hours). The 289 survey takers displayed char- 
acteristics similar to the general student popula- 
tion regarding the distribution of sex, disability, 
ethnicity, and age characteristics. The small num- 
ber of ENGH 0990 students who had taken ENGH 
0890 previously were excluded from any statisti- 
cal analyses to eliminate any confounding factors 
when analyzing tests scores as possibly predictive 
of final grades. 

Regarding the college’s placement system that 
created the ENGH 0890 and 0990 classes, a score 
of 19 or more on both the ACT English and read- 
ing sections exempted students altogether from 
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ENGH 0890 and ENGH 0990. Lower or absent 
ACT score(s) meant a student would complete two 
multiple-choice tests: ACT'S COMPASS writing test 
(an editing or grammar test) and the Degrees of 
Reading Power test (DRP, a reading test). A 79 or 
lower score on the COMPASS and a 76 or lower 
score on the DRP defined placement for ENGH 
0990. A49 or lower score on the COMPASS and a 
69 or lower score on the DRP defined placement 
for ENGH 0890. 

Design 

For the current study, 12 experienced English 
teachers holistically scored the 431 timed essays 
written by in-coming freshmen during the first 
week of classes. These students were in one of two 
developmental English courses: ENGH 0890, Fun- 
damental English Skills and ENGH, 0990, College 
Preparatory Writing. The students addressed one 
of two prompts and were allowed 30 minutes to 
write an essay. One prompt asked students to write 
about reasons for attending college. The other 
asked for comparisons between living in a small 
town, big city, and/or rural area. More students 
addressed the former rather than the latter 
prompt. Exam instructions that all teachers read 
to students encouraged them to organize their 
essays by asking them to “plan. ..prewrite, write, 
and revise" essays, suggesting that they “may in- 
clude an outline or prewriting" with their final 
copy, and encouraging them to “include examples 
and specifics" in that copy, too. 

A student’s motivation to write his or her best 
may have been affected by his or her current en- 
rollment in ENGH 0890 or ENGH 0990. The 
ENGH 0890 teachers read to their students, “If 
you do well on this essay, you may advance to 
ENGH 0990." After testing, 32 students were given 
that opportunity. All English teachers read this to 
all students, “How well or poorly you write may 
affect my teaching this class." Doing poorly on the 
timed-essay exam, students were told, may mean 
more grammar instruction. Doing well may mean 
more college-level instruction. To try to motivate 
individual writers, teachers also stated, “Your writ- 
ing may affect how I teach you as an individual 
writer.” 

Procedures 

The 12 English teachers scored the resulting 
431 timed essays by using a rubric. The 6-point 
rubric was developed and pilot-tested during the 
previous semester when two teachers— with over 
12 years of holistic-scoring experience between 
them— had asked 140 ENGH 0890 and 0990 stu- 
dents to write timed essays. The teachers applied 
the SAT II Scale to those essays and revised or 
adapted it to be more descriptive of their students’ 
writing (see Wolcott & Legg, 1998; contact JDE 
for rubric). Anchor and range-finder essays were 
selected from among the 140 essays: An “anchor” 
essay represents the typical or average essay for all 


essays receiving a particular score; whereas, a 
“range finder” essay represents the essays identi- 
fied as in between two scores. Through training 
with anchors and range finders, English teachers 
learn to apply the rubric so that the two scores 
given any essay most often agree. In December 
2002, English teachers completed a 3-hour train- 
ing period, and, in January 2003, completed a 2 
1/2-hour training period, immediately prior to 
scoring the 431 timed essays in this study. 

Each essay was read and scored by two En- 
glish raters/teachers. Regarding 97% of essays, the 
two scores were identical or within one ordinal of 
one another. When the two scores disagreed, a 
veteran rater resolved this “split.” Raters did not 
score any essay as a six, perhaps because neither 
essay prompt solicited enough narrative or descrip- 
tive writing and both solicited more expository 
writing. During a large-scale assessment of timed 
essays, researchers Engelhard, Gordon, and 
Gabrielson (1992) “found that the mode of dis- 
course was a significant factor in predicting writ- 
ing quality [in timed essays]: narrative writing tasks 
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received the highest scores, with descriptive tasks 
next and expository tasks last" (Wolcott & Legg, 
1998, p. 15). Another explanation for why no stu- 
dent scored a six may be found related to this fact: 
All the students, prior to responding to either 
prompt, had been labeled as developmental or 
remedial by the placement system. This aspect may 
have affected the English teachers' rating the es- 
says as well as the motivations or efforts of the 
students writing the essays. 

Data Analysis 

For the purpose of data analysis, the timed- 
essay scores created a standard against which these 
multiple-choice tests were measured. A score of 
2.5 or more would have placed a student into 
ENGH 0990, and a score of 2.0 or less would have 
placed a student into ENGH 0890. Data analysis 
also included some items on a survey for the basic 
writing program. These items asked students about 
their completing assignments, their attending class, 
and their possibly seeking help at the Writing Cen- 
ter. Students were also asked if they had experi- 
enced a nonacademic adverse event that impeded 
their academic progress. And students were asked, 
too, if they had tried their best when writing the 
timed essay given when the semester began. Sur- 
veys were administered at semester’s end. 

Regression analysis was conducted to deter- 
mine if the holistically scored essay was predictive 


of grades in the course when controlling for lab 
use, class attendance, assignment completion, and 
external adverse events. If predictive, the analysis 
would provide support for the argument that the 
timed essay offered additional information on stu- 
dents' writing ability useful for placement above 
and beyond the standard use of multiple-choice 
tests currently used. Regarding data analysis, the 
regression analysis and its underlying assumptions 
followed established guidelines (Siegel, 1997). The 
regression analysis included normal probability 
plots for the residuals and incorporated scatter 
plots for each independent and dependent vari- 
able and for the standardized residuals. The de- 
pendent variable — grade point average (GPA)— was 
squared to achieve normality. No outliers of any 
concern affected the regression analysis. Indepen- 
dent variables, regarding the survey, were changed 
to dummy variables due to a lack of normality as 
follow. Infrequent Lab Use: 1= never or once, 0 = 
more frequent use; Class Attendance: 1 = all classes, 
0 = less frequent attendance; Assignment Comple- 
tion: 1= all assignments, 0 = missing assignments; 
Effort on Timed Essay: 1 = best effort on survey, 0 
= other level of effort; and; Adverse Event(s): 1= 
strongly agree or agree that outside experiences 
(personal, health, family, or financial difficulties) 
negatively affected academic progress, 0 = disagree 
or strongly disagree. Another dummy variable, 
credit hours (1 = 12+ credits before the English 
course, 0 = 11- credit hours), was added to the re- 
gression analysis to control for the effect of other 
course work on student performance. 

T-tests were also applied to data and were 
used to search for any significant differences be- 
tween all the students placed into ENGH 0890 and 
0990 as well as differences among students in 
ENGH 0890 and among those in ENGH 0990. The 
level of significance was set at p < .05. When the 
Levene’s test for equality of variances was signifi- 
cant, a nonparametric analog to the f-test formed 
the analysis. 

Findings 

Standardized Tests and Timed Essays 

The following descriptive statistics present 
student placement in a matrix comparing place- 
ment using COMPASS writing scores only versus 
the scored essay. Substantial disagreement exists 
between these two tests (see Table 1, p. 6). With 
the timed essay as the standard for placement into 
ENGH 0890 and 0990, COMPASS only correctly 
places 62% of students. This “miss-placement” is 
greater in ENGH 0890. When Table 1 is recreated 
with only students who self-reported their “best 
effort" on the timed essay (n = 62), the percentage 
of students correctly placed by the current system 
is 65%, a small change from the original 62%. 

Pearson correlations also support the asser- 
tion that the COMPASS, DRP, and timed essay are 

continued on page 6 


4 


Journal of Developmental Education 


Volume 28, Number 1, Fall 2004 


5 


Table 1 


Number of Students Placed in Courses by Instrument (n = 290) 


Instrument 


Course Placement 



ENGH 0890 

% of Total 

ENGH 0990 

% of Total 

COMPASS writing test 

117 

40.3% 

173 

59.7% 

Holistically scored essay 

56 

19.3% 

234 

80.7% 

Both instruments in agreement 

31 

10.7% 

148 

51.0% 

Incorrectly placed by COMPASS 

86 

29.7% 

25 

8.6% 


Note: The holistically scored essay and faculty judgment formed the standard, so a timed 
essay score of 2.5 or more meant ENGH 0990 placement. Regarding COMPASS, a score of 
79 or less meant ENGH 0990 placement and 49 or less meant ENGH 0890 placement. 


continued from page 4 

not measuring equivalent student aptitudes. The 
correlation between the timed-essay and COM- 
PASS scores is .29 ( n - 271). The correlation be- 
tween timed-essay and DRP scores is .29 ( n = 255), 
a moderately weak correlation. A moderate cor- 
relation exists between COMPASS and DRP scores 
R = .49 (n = 361); however, this correlation is be- 
tween two tests that are indirect measures of stu- 
dent writing. 

With the timed-essay score as a standard, re- 
searchers considered the students who took ENGH 
0990 when their timed-essay score(s) indicated a 
ENGH 0890 placement. If a timed-essay score was 
a better placement instrument than the standard- 
ized test scores, researchers would expect that these 
ENGH 0990 students— who should have been 
placed into ENGH 0890 according to timed-essays 
scores— would have lower grades in ENGH 0990 
as compared to their classmates who had timed- 
essays scores confirming their original placement 
into ENGH 0990. This proved to be the case. The 
incorrectly placed ENGH 0990 students had lower 
final grades when compared to the correctly 
placed ENGH 0990 students. The former group's 
average GPA was 2.2 (a “C” average, n = 22) and 
the latter group's average GPA was 2.8 (a “B-” av- 
erage, n - 117). 

Ironically, as the semester started and the 
English teachers offered 32 ENGH 0890 students 
opportunities to enroll in ENGH 0990 courses 
because of relatively high timed-essay scores, only 
12 (38%) decided to make the change (and not 
have to pay for ENGH 0890, a 5-credit course). 
The average GPA of the 12 students who moved 
from ENGH 0890 to ENGH 0990 and who were 
also taking their first English course was 2.7 ( n = 
12) compared to a 2.8 GPA (n - 117) for the ENGH 
0990 students who were accurately placed accord- 
ing to their timed-essay score (and COMPASS 
score). It is interesting that the “moved” students 
performed relatively well in ENGH 0990, even in 
light of the small sample size. 

These results do not indicate the need to 
merely readjust the cut scores for placement us- 
ing the multiple choice tests, as cut scores have 
been set to achieve an 80% probability of a “C" 
and a 60% probability of a “B” in the courses us- 


ing standard logistic regression techniques and the 
information from the placement tests (Noble & 
Sawyer, 1997; Utah Valley State College Office of 
Institutional Research, 1997). The holistically 
scored essays provide additional information on 
students’ writing ability not present in the mul- 
tiple-choice placement tests. 

Perception, Motivation, Ethnicity, and 
Survey 

According to survey results, many students 
who completed either ENGH 0890 or 0990 did 
not believe that they were appropriately placed. 
Approximately 21% of ENGH 0890 students re- 
ported that they should have been placed in a 
higher level course, ENGH 0990 or ENGL 1010 
(freshman composition). For 83% of them, their 
perceptions were supported by either a favorable 
essay or COMPASS score. Only .4% of the ENGH 
0990 students believed that they should have been 
in the lower course, ENGH 0890; however, approxi- 
mately 13% of ENGH 0990 students thought that 
they should have been placed into ENGL 1010. 
Approximately 21% of ENGH 0890 students and 
16% of ENGH 0990 students had no opinion on 
their English placement. 

Regarding other survey results, approxi- 
mately 34% of ENGH 0890 students and 46% of 
ENGH 0990 students have self-reported giving 
their best effort on the timed essay. Motivation 
(or self-confidence) appears to be higher among 
ENGH 0990 stu- 
dents, perhaps 
because they 
were already in 
ENGH 0990 
when taking the 
timed-essay 
exam. In con- 
trast, in spite of 
the timed essay 
representing a 
real opportu- 
nity to move 
into ENGH 
0990, ENGH 
0890 students 


may have been disheartened by their having been 
placed into ENGH 0890 by the DRP and COM- 
PASS or ACT tests. 

Table 2, through a regression analysis, dis- 
plays the effects of several variables on student 
GPA in ENGH 0890 and ENGH 0990, including 
student essay scores. As indicated by the standard- 
ized regression coefficients, the most influential 
variable on students' final grades is the comple- 
tion of assignments. Only 42% of ENGH 0890 stu- 
dents and only 53% of ENGH 0990 students have 
reported completing all of their assignments. A 
large decline in grades for these courses is found 
among the students who report completing less 
than all assignments. This finding may suggest that 
many basic writers need help with their time-man- 
agement skills, educational values, or academic 
habits. The second most influential variable on 
course grade is having adverse events— personal, 
health, family, or financial difficulties— outside of 
class that negatively affect academic progress (see 
Table 2). 

These factors are followed by students' scores 
on timed essays and their frequency of visiting the 
Writing Center. The significance of timed-essay 
scores may be interpreted as more evidence sup- 
porting their use for placement purposes. Using 
the Writing Center may be interpreted as some 
basic writers realizing their own needs for instruc- 
tional assistance to improve their writing skills and 
their grades. About 47% of ENGH 0890 students 
and 69% of ENGH 0990 students report attend- 
ing the Writing Center two or more times that 
semester. 

Class attendance is not statistically significant 
when considered in conjunction with the other 
variables in the model: 97% of ENGH 0890 stu- 
dents and 96% of ENGH 0990 students report at- 
tending almost every class meeting. In addition, 
effort on the timed essay and completing 12+ credit 
hours prior to ENGH 0890 or ENGH 0990 are 
not statistically significant predictors. 

Disparate impact is assessed using f-tests com- 
paring performance on the essay test and multiple- 

continued on page 8 


Table 2 

Regression Equation Predicting Academic Success 

Predictor 

Regression 

Standard 

Standardized 

T- 

Significance 

Variables 

Coefficients 

Error 

Coefficients 

Statistic 

Level 

Infrequent lab use 

-1.355 

.664 

-.148 

-2.040 

.043 

Class attendance 

.168 

.720 

.017 

.233 

.816 

Assignment 

3.434 

.705 

.389 

4.871 

.000 

completion 






Effort 

.342 

.694 

.038 

.493 

.623 

Adverse events 

-2.128 

.628 

-.241 

-3.387 

.001 

Credit hours 

-.09702 

.654 

-.011 

-.148 

.882 

Holistic score 






(timed essay) 

1.003 

.480 

.149 

2.091 

.038 

Constant 

6.239 

1.591 


3.922 

.000 

Note, timed-essay scores for ENGH 0890 students ranged from 1 to 4 and for ENGH 
0990 students, from 1 to 5. 
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choice tests (see Table 3). T-tests show no signifi- 
cant differences among student subpopulations 
regarding the holistically scored, timed essay. How- 
ever, minority students have significantly lower 
scores on both the COMPASS writing test and DRP 
reading test when compared with nonminority (or 
white) students. The timed-essay exam may have 
less of a disparate impact on minority students 
because that exam, unlike the multiple-choice 
COMPASS and DRP tests, provides students with 
an opportunity to produce and use their own writ- 
ten language. In comparison, an underlying as- 
sumption of a multiple-choice test is that students 


should know a standardized or ideal English lan- 
guage, one that exists unaffected by genre con- 
ventions, subject matter, or discourse communi- 
ties. 

Implications 

Considering that national reports indicate 
generally poor composition instruction across the 
country at the high school level (Basinger, 2000; 
Kellogg, 2001; Potter, 2003; Rooney, 2003; Strauss, 
2003), accurate placement into composition 
courses at college seems critical. This research in- 
dicates that an essay exam is valuable for place- 
ment purposes. In the judgement of experienced 
English faculty members, many students are cur- 
rently misplaced into writing courses by reliance 
solely on multiple-choice tests. Descriptive statis- 
tics show that students, who should be placed in 
ENGH 0990 versus ENGH 0890 (according to the 
holistic essay), perform well when advanced to the 
higher level course. Students, who should have 
been placed in the ENGH 0890 versus ENGH 0990 
(again using the essay score), earn substantially 
lower grades in the more advanced course. The 
holistic score also significantly contributes to the 
prediction of grades in writing courses when con- 
trolling for lab use, class attendance, assignment 


completion, effort, adverse events, and credit 
hours taken during the term. 

This study suggests that accurate placement 
may improve with a timed-essay score. To be clear, 
if only a single means of placement could be se- 
lected, and one had to choose between a reading 
test, an editing test, or a timed essay, a timed essay 
seems preferable. But, as findings also demon- 
strate, a timed-essay score may be used as a pri- 
mary means for placement and a reading test score 
as a supplemental consideration so that combin- 
ing the measures further improves accurate place- 
ment. 

The benefits of more accurate placement can 
be substantial. First, accurate placement and suc- 
cessful completion of 
basic writing programs 
have been linked to im- 
proved student reten- 
tion (Baker & Jolly, 
1999; Boylan, 1999; 
Cunningham, 1983; 
Glau, 1996; Kiefer, 
1983; McGregor & 
Attinasi, 1996; White, 
1995). Second, the 
timed-essay exam shows 
less of a disparate im- 
pact on minority stu- 
dents as compared to 
placement with the mul- 
tiple-choice tests. Third, 
improved placement 
may lead to more effec- 
tive pedagogy because an English teacher, rather 
than attempting to address the needs of basic writ- 
ers whose skills range from very underprepared 
to prepared for college writing, may focus on teach- 
ing a class of students who have similar abilities 
and needs as writers. A fourth way that placement 
by timed essay may be effectively linked to a basic 
writing program would be for the English teach- 
ers to have the placement timed-essay function as 
a pretest, too. They could later ask basic writers 
to write a second timed essay, a posttest, to end a 
course (Drakulich, 1979; Glassman, 1983; Mills, 
2000; Resch, 1972; Schaumburg, 1975; Strugala, 
1983). Pre- and posttest scores could then be com- 
pared, considered, and discussed prior to making 
curricular modifications. Finally, such practices 
may make placement and assessment easier to ex- 
plain to students, motivate students more, and cre- 
ate new benefits for students like a long-term com- 
mitment to writing improvement. Reading and 
rating the timed-essays can also become a process 
for the development of basic writing faculty. In 
short, use of a timed-essay in the basic composi- 
tion program has the potential to increase the shar- 
ing of educational goals with students as well as 
among faculty. 

In the current study, accurate placement not 
only affected the effectiveness of curriculum and 


pedagogy (i.e., outcome assessment) but also so- 
licited policy decisions. One such decision was 
deciding the cut-off scores relevant to timed es- 
says for ENGH 0890 and ENGH 0990. To explain, 
the faculty in the Basic Composition Department 
identified the cut-off score, and their decision was 
informed by their research, by institutional re- 
search into matriculation from ENGH 0890 to 
0990, and by their curricular goals and experiences 
as writing teachers. Consequently, with an im- 
proved stratification of basic writers, a low cut-off 
score would concentrate only the weakest basic 
writers into fewer ENGH 0890 sections, and a 
majority of basic writers would only have to com- 
plete ENGH 0990 prior to Freshman Composition. 
Knowing that the weakest basic writers need more 
instruction and one-on-one help, the teachers 
hoped that perhaps administrators outside of the 
Basic Composition Department would at least rec- 
ognize the need for a smaller class size for ENGH 
0890 if not also for ENGH 0990. 

Considering this study— in which timed-essay 
scores, final course grades, and questionnaire re- 
sults were linked— the English teachers now have 
the opportunity to identify specific basic writers 
needing help prior to their earning a low final 
grade. The English teachers may want to monitor 
each basic writer’s progress, identify those not 
completing assignments, and intervene not only 
as teachers but also as advisors. Teachers, too, may 
want to require some students to use the Writing 
Center in addition to encouraging all students to 
go. Also, teachers may want to identify the basic 
writers who are experiencing nonacademic adverse 
events that affect academic progress and refer 
them to the appropriate offices. To facilitate this 
interventionist philosophy, however, the English 
teachers would need more one-on-one contact with 
students. Do the teachers have the time? Class size 
may discourage that level of involvement because 
administration has admitted 26 students in a sec- 
tion of ENGH 0890 or ENGH 0990 as a goal. Be- 
cause of the labor-intensive nature of teaching 
writing, the National Council of English Teachers 
(NCTE) suggests, “No more than 20 students 
should be permitted in any writing class" and “re- 
medial or developmental sections should be lim- 
ited to a maximum of 15 students" (National Coun- 
cil of Teachers of English, 1987, p. 7). 

Considering that some policy decisions are 
impossible for the Basic Composition Department 
to make alone, hopefully the college administra- 
tion will take advantage of the current and forth- 
coming research findings for course placement. 
The English and Literature Department needs to 
be involved, too. That Department decides student 
placement into an entry-level writing course, ENGL 
1010, and is currently using the multiple-choice 
reading and editing tests for placement. Because 
of ongoing discussions, however, the English and 
Literature Department may support a new place- 
continued on page 10 


Table 3 

Disparate Impact of COMPASS and DRP Tests Versus 
Timed-Essay Exams 


Student Average Scores 


Subpopulations 

Holistic 

score 

n 

COMPASS 

score 

n 

DRP 

score 

n 

Full time 

2.8 

186 

54 

245 

78 

230 

Part time 

2.8 

89 

54 

140 

78 

132 

Minority 

2.8 

35 

48* 

56 

73* 

55 

Nonminority 

2.8 

237 

55* 

329 

79* 

307 

Female 

2.9 

115 

54 

161 

75* 

148 

Male 

2.8 

157 

55 

224 

81* 

214 

Disabled 

2.6 

20 

46 

24 

74 

24 

Nondisabled 

2.9 

252 

55 

361 

79 

338 

Traditional 

2.8 

51 

55 

81 

80 

81 

Nontraditional 

2.9 

221 

54 

304 

78 

281 


Note. Significant difference is defined as p < .05. 
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merit system. Still, whereas English teachers and 
researchers may see the interests of students and 
the college converging regarding the need for 
better placement into all entry-level writing courses, 
administrators may focus on the current standard- 
ized placement tests as cost-effective. Students, in 
turn, may focus on paying for an “extra" writing 
course(s), with either ENGH 0890 and/or 0990 
both possibly being prerequisites for freshman 
composition because of the placement system. Or 
students may focus on enrolling in a course, fresh- 
man composition, when a basic writing course is 
needed as the first writing course in college. Seem- 
ingly, if the administration is not an agent for 
change, the existing problematic situation may 
continue and the chance for meaningful policy 
changes may pass. 

Future Research Considerations 

Because this study does not offer clear sup- 
port for using only standardized tests or only a 
holistically scored essay for placement, broad re- 
search, not just replication, seems to be needed. 
Besides replicating the current research with larger 
numbers of students, the college may support and 
study placement into ENGH 0890 and 0990 (and 
ENGL 1010) by using timed-essay exams alone or 
by combining a timed-essay exam score with a DRP 
score. Motivational issues, for instance, may be bet- 
ter addressed by asking selected in-coming fresh- 
men to participate in a pilot study, in which place- 
ment into ENGH 0890, ENGH 0990, or ENGL 
1010 is determined by a timed-essay exam only. 
These students may also be allowed more than 30 
minutes to write their essay to lessen some test 
anxiety and increase the essay exam’s validity. 

Fortunately, in September 2003, college re- 
searchers have already begun to involve greater 
numbers of students in placement research by scor- 
ing timed essays written by approximately 300 stu- 
dents enrolled in ENGH 0890, ENGH 0990, and 
ENGL 1010. The timed essays will be scored by 
English faculty as well as scored electronically by 
ACT’S e-Write program. Besides replicating prior 
research— correlating DRP, COMPASS, and timed- 
essay scores— researchers will be in a better posi- 
tion to suggest how placement by timed-essay may 
be implemented accurately and fairly. 

The study reported herein reaches conclu- 
sions supported by previously cited longitudinal 
studies, some involving all incoming freshmen or 
all students in composition courses. So, rather than 
research more into a question that has been affir- 
matively answered, “Are timed-essay exams pref- 
erable to multiple-choice tests if only one, afford- 
able measure is going to be used for placement?”, 
researchers should explore and test how to imple- 
ment a fairer placement system based on a holisti- 
cally scored essay as the primary means for place- 
ment. The authors' new efforts have involved more 


administrative stakeholders on campus so that the 
political will to improve composition placement 
may exist at their institution. Colleagues across 
various geographic locations and institutions hope- 
fully will contribute to another level of research 
on these issues — how best to implement placement 
by a holistically timed essay, possibly in combina- 
tion with another secondary means of placement- 
— so that basic writers may be better identified and 
taught. 
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5, for more information. 
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10-12, 2004-North Carolina Community College Instructors’ Conference, 
“The Wheels of Learning,” at the Koury Convention Center in Greens- 
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